本文介绍了信息性多臂强盗(IMAB)模型,在每个回合中,玩家选择手臂,观察符号,并以符号的自我信息形式获得未观察到的奖励。因此,手臂的预期奖励是产生其符号的源质量函数的香农熵。玩家的目标是最大程度地提高与武器的熵值相关的预期奖励。在假设字母大小是已知的假设下,为IMAB模型提出了两种基于UCB的算法,该算法考虑了插件熵估计器的偏差。第一种算法在熵估计中乐观地纠正了偏置项。第二算法依赖于数据依赖性置信区间,该置信区间适应具有较小熵值的源。性能保证是通过上限为每种算法的预期遗憾提供的。此外,在Bernoulli案例中,将这些算法的渐近行为与伪遗憾的Lai-Robbins的下限进行了比较。此外,在假设\ textit {cract}字母大小的假设下是未知的,而播放器仅知道其上方的宽度上限,提出了一种基于UCB的算法,在其中,玩家的目的是减少由该算法造成的遗憾。未知的字母尺寸在有限的时间方面。数字结果说明了论文中介绍的算法的预期遗憾。
translated by 谷歌翻译
我们考虑了二进制隐藏的马尔可夫模型上的高维平均值估计问题,该模型阐明了数据,样本大小,维度和统计推断中信号强度的记忆之间的相互作用。在此模型中,估算器观察$ n $样品的$ d $ dimensional参数vector $ \ theta _ {*} \ in \ mathbb {r}^{d} $,乘以随机符号$ s_i $($ 1 \ \ $ 1 \ \ s_i $) le i \ le n $),并被各向同性标准高斯噪声损坏。标志$ \ {s_ {i} \} _ {i \ in [n]} \ in \ { - 1,1 \}^{n} $是从带有flip概率$ \ flip概率$ \的固定同质马尔可夫链中绘制的delta \ in [0,1/2] $。随着$ \ delta $的变化,该型号顺利地插入了两个认真的模型:高斯定位模型,$ \ delta = 0 $和高斯混合模型,$ \ delta = 1/2 $。假设估算器知道$ \ delta $,我们建立了一个几乎最小的最佳(达到对数因素)估计错误率,作为$ \ | \ theta _ {*} \ |,\ delta,d,d,n $的函数。然后,我们为估计$ \ delta $的情况提供了上限,假设$ \ theta _ {*} $的知识(可能不准确)。当$ \ theta _ {*} $是一个准确已知的常数时,界限被证明是紧身的。然后将这些结果组合到算法中,该算法用$ \ delta $ unknown估算$ \ theta _ {*} $先验,并说明了其错误的理论保证。
translated by 谷歌翻译
For applications that require processing large amounts of text at inference time, Large Language Models (LLMs) are handicapped by their limited context windows, which are typically 2048 tokens. In-context learning, an emergent phenomenon in LLMs in sizes above a certain parameter threshold, constitutes one significant example because it can only leverage training examples that fit into the context window. Existing efforts to address the context window limitation involve training specialized architectures, which tend to be smaller than the sizes in which in-context learning manifests due to the memory footprint of processing long texts. We present Parallel Context Windows (PCW), a method that alleviates the context window restriction for any off-the-shelf LLM without further training. The key to the approach is to carve a long context into chunks (``windows'') that fit within the architecture, restrict the attention mechanism to apply only within each window, and re-use the positional embeddings among the windows. We test the PCW approach on in-context learning with models that range in size between 750 million and 178 billion parameters, and show substantial improvements for tasks with diverse input and output spaces. Our results motivate further investigation of Parallel Context Windows as a method for applying off-the-shelf LLMs in other settings that require long text sequences.
translated by 谷歌翻译
Are extralinguistic signals such as image pixels crucial for inducing constituency grammars? While past work has shown substantial gains from multimodal cues, we investigate whether such gains persist in the presence of rich information from large language models (LLMs). We find that our approach, LLM-based C-PCFG (LC-PCFG), outperforms previous multi-modal methods on the task of unsupervised constituency parsing, achieving state-of-the-art performance on a variety of datasets. Moreover, LC-PCFG results in an over 50% reduction in parameter count, and speedups in training time of 1.7x for image-aided models and more than 5x for video-aided models, respectively. These results challenge the notion that extralinguistic signals such as image pixels are needed for unsupervised grammar induction, and point to the need for better text-only baselines in evaluating the need of multi-modality for the task.
translated by 谷歌翻译
Diffusion models have achieved great success in modeling continuous data modalities such as images, audio, and video, but have seen limited use in discrete domains such as language. Recent attempts to adapt diffusion to language have presented diffusion as an alternative to autoregressive language generation. We instead view diffusion as a complementary method that can augment the generative capabilities of existing pre-trained language models. We demonstrate that continuous diffusion models can be learned in the latent space of a pre-trained encoder-decoder model, enabling us to sample continuous latent representations that can be decoded into natural language with the pre-trained decoder. We show that our latent diffusion models are more effective at sampling novel text from data distributions than a strong autoregressive baseline and also enable controllable generation.
translated by 谷歌翻译
Hebrew is a Morphological rich language, making its modeling harder than simpler language. Recent developments such as Transformers in general and Bert in particular opened a path for Hebrew models that reach SOTA results, not falling short from other non-MRL languages. We explore the cutting edge in this field performing style transfer, text generation and classification over news articles collected from online archives. Furthermore, the news portals that feed our collective consciousness are an interesting corpus to study, as their analysis and tracing might reveal insights about our society and discourse.
translated by 谷歌翻译
A gradual semantics takes a weighted argumentation framework as input and outputs a final acceptability degree for each argument, with different semantics performing the computation in different manners. In this work, we consider the problem of attack inference. That is, given a gradual semantics, a set of arguments with associated initial weights, and the final desirable acceptability degrees associated with each argument, we seek to determine whether there is a set of attacks on those arguments such that we can obtain these acceptability degrees. The main contribution of our work is to demonstrate that the associated decision problem, i.e., whether a set of attacks can exist which allows the final acceptability degrees to occur for given initial weights, is NP-complete for the weighted h-categoriser and cardinality-based semantics, and is polynomial for the weighted max-based semantics, even for the complete version of the problem (where all initial weights and final acceptability degrees are known). We then briefly discuss how this decision problem can be modified to find the attacks themselves and conclude by examining the partial problem where not all initial weights or final acceptability degrees may be known.
translated by 谷歌翻译
自动驾驶汽车必须能够可靠地处理不利的天气条件(例如,雪地)安全运行。在本文中,我们研究了以不利条件捕获的转动传感器输入(即图像)的想法,将其下游任务(例如,语义分割)可以达到高精度。先前的工作主要将其作为未配对的图像到图像翻译问题,因为缺乏在完全相同的相机姿势和语义布局下捕获的配对图像。虽然没有完美对准的图像,但可以轻松获得粗配上的图像。例如,许多人每天在好天气和不利的天气中驾驶相同的路线;因此,在近距离GPS位置捕获的图像可以形成一对。尽管来自重复遍历的数据不太可能捕获相同的前景对象,但我们认为它们提供了丰富的上下文信息来监督图像翻译模型。为此,我们提出了一个新颖的训练目标,利用了粗糙的图像对。我们表明,我们与一致的训练方案可提高更好的图像翻译质量和改进的下游任务,例如语义分割,单眼深度估计和视觉定位。
translated by 谷歌翻译
这项研究提出了一种检测近距离红外(NIR)眼周眼图像的酒精消耗的方法。该研究的重点是确定外部因素(例如酒精对中枢神经系统(CNS))的影响。目的是分析这如何影响虹膜和学生运动,以及是否可以使用标准的Iris NIR相机捕获这些更改。本文提出了一个新型的融合胶囊网络(F-CAPSNET),以对饮酒受试者拍摄的虹膜NIR图像进行分类。结果表明,使用一半参数作为标准胶囊网络算法,F-CAPSNET算法可以检测IRIS NIR图像中的酒精消耗,精度为92.3%。这项工作是开发自动系统以估计“适合值班”并防止因饮酒而导致事故的一步。
translated by 谷歌翻译
联合学习(FL)是使用Edge设备上可能可用的私人数据训练机器学习模型的新兴范式。 FL的分布式操作引起了集中式机器学习中未遇到的挑战,包括需要保留本地数据集的隐私以及由于重复交换更新模型而导致的通信负载。这些挑战通常通过引起更新模型的某些失真的技术来单独解决,例如当地差异隐私(LDP)机制和有损压缩。在这项工作中,我们提出了一种方法创造的联合隐私增强和量化(JOPEQ),该隐私和量化共同实现了FL环境中的有损压缩和隐私增强。特别是,Jopeq利用基于随机晶格的矢量量化,这是一种通用压缩技术,其副产品失真在统计学上等同于加性噪声。通过使用专用的多元隐私保护噪声来增强模型更新,可以利用这种失真来增强隐私。我们表明,JOPEQ在持有所需的隐私级别的同时,根据所需的比特率同时量化数据,而不会特别影响学习模型的实用性。这是通过分析的LDP保证,失真和收敛范围的推导以及数值研究所示的。最后,我们从经验上断言,乔普克(Jopeq)拆除了已知的普通攻击,以利用隐私泄漏。
translated by 谷歌翻译